Add Rainy Mood plugin provider by jlpouffier · Pull Request #3844 · music-assistant/server

jlpouffier · 2026-05-06T11:34:11Z

Summary

Adds a new experimental Rainy Mood plugin provider that mixes looping ambient rain audio from rainymood.com transparently into whatever the player's queue is playing — without touching the queue itself.

Generalises the plugin/audio-overlay surface area so any plugin can declare itself as an audio overlay source.

Dependency

Requires Add AUDIO_OVERLAY to ProviderFeature enum models#221 (merged, shipped in music-assistant-models 1.1.118)

Demo

Watch it live here

How it works

Setting

A single Rain Volume Ratio (%) config entry controls how loud the rain is relative to the music:

Value	Effect
0 %	Rain inaudible — music plays normally
100 %	Rain as loud as the music (default)
200 %	Rain twice as loud as the music

Behaviour

Situation	What happens
Enable while playing	Rain starts immediately — stream is restarted from current position to pick up the overlay
Enable while paused / stopped	Rain is armed but the stream is not restarted — rain will be present when playback resumes
Disable while playing	Rain stops immediately — stream is restarted cleanly
Disable while paused / stopped	Rain is disarmed silently
Track transition (next / end of track)	Rain continues seamlessly — the persistent rain FFmpeg process is not restarted
Seek	Rain continues seamlessly — same process, no restart
Pause	Rain pauses naturally (the mix FFmpeg stops outputting when its music input is idle)
Resume after pause	Rain resumes from where it left off
Queue emptied or finished	Rainy Mood auto-disables. This is an intentional design decision: Rainy Mood is a per listening experience setting. You start a session, enable the rain, fall asleep — the next morning the queue is done and you start fresh with a clean slate, no rain lingering from the night before.

Architecture

Generic overlay interface on PluginProvider:

ProviderFeature.AUDIO_OVERLAY (added in models#221) lets any plugin declare overlay capability.
PluginProvider.is_overlay_active(player_id) -> bool — cheap predicate the controller polls per player.
PluginProvider.get_overlay_stream(player_id, pcm_format) -> AsyncGenerator[bytes, None] | None — returns a volume-adjusted PCM stream in the requested format, so the plugin handles its own conversion/scaling.

The streams controller uses mass.get_providers_supporting_feature(ProviderFeature.AUDIO_OVERLAY) to discover all candidates and mixes every active overlay together, not just the first match.

Mixing via FFmpeg amix:

A new mix_pcm_streams(inputs, pcm_format) helper in helpers/ffmpeg.py spawns a dedicated FFmpeg subprocess with one -i pipe:<fd> per input and -filter_complex "...amix=inputs=N:duration=first:normalize=0[mix]". Inputs are fed via asyncio.StreamWriter with proper drain() backpressure so the event loop is not starved when source generators yield bursty chunks. duration=first binds the mix length to the music input (so a looping overlay never extends playback) and normalize=0 prevents the per-input volume attenuation that amix does by default.

Rainy Mood specifics:

A persistent RainBuffer FFmpeg subprocess runs per player for as long as rain is active, outputting raw PCM in whatever format the player's pipeline asks for (via ensure_format). Because it stays alive across track boundaries, rain content never resets between songs or on seek.
The plugin's scaled_stream applies the per-instance volume ratio before yielding rain PCM into the mixer.

Coverage:

Overlay injection covers all three audio paths via wrap_overlay_if_active: serve_queue_item_stream, serve_queue_flow_stream (HTTP — used by Sonos native), and get_stream (direct PCM — used by Sendspin / AirPlay).

The overlay lookup always uses queue_id (the queue owner) rather than player_id, so it works correctly when a player delegates transport to a bridge protocol (e.g. a Sonos speaker streaming via Sendspin).

Test plan

🤖 Generated with Claude Code

github-actions · 2026-05-06T11:35:16Z

🔒 Dependency Security Report

✅ No dependency changes detected in this PR.

marcelveldt · 2026-05-06T20:06:11Z

+                    return cast("tuple[Any, float]", overlay)
+        return None
+
+    async def apply_rain_overlay(


this should be more generic / reusable

rain is the actual implementation (covered in the plugin) so the global implementation would be optional overlay support

global function that gets the active overlay (returned as string for an ID)
global function that routes the request to the right plugin based on that id

ProviderFeature.AUDIO_OVERLAY added to models

marcelveldt · 2026-05-06T20:08:40Z

+        music_gen: AsyncGenerator[bytes, None],
+        rain_reader: Any,
+        rain_vol: float,
+        pcm_format: AudioFormat,


this assumes that the pcm format of the audio and audio overlay is the same which it wont be

@marcelveldt We should consider using ffmpeg filters here instead of calculating this ourselves. I think something like amix might be able to do what we want.

Addresses two review comments on music-assistant#3844: 1. **Generic overlay interface**: Replace duck-typed `get_player_overlay()` with a proper `ProviderFeature.AUDIO_OVERLAY` capability flag. Any plugin can now declare overlay support and implement `is_overlay_active()` / `get_overlay_stream()`. The streams controller looks up providers by feature flag rather than by rain-specific method name. 2. **PCM format propagation**: `get_overlay_stream(player_id, pcm_format)` receives the exact format used by the music stream. The `RainBuffer` configures FFmpeg to match, so sample rate, bit depth and channel count are always in sync. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

MarvinSchenkel · 2026-05-13T10:48:36Z

+    # Audio overlay support (used by Rain Mood and similar plugins)
+    # ------------------------------------------------------------------
+
+    def get_active_overlay_provider(self, player_id: str) -> PluginProvider | None:


What should happen if multiple overlays are active?

MarvinSchenkel · 2026-05-13T10:48:54Z

+        :param player_id: The player to check.
+        :returns: PluginProvider instance, or None.
+        """
+        for prov in self.mass.providers:


You can use mass.get_providers_supporting_feature here instead

Adds a new experimental plugin that mixes looping ambient rain audio from rainymood.com transparently into whatever is playing in the queue, without touching the queue itself. - New `providers/rain_mood/` plugin with a persistent per-player FFmpeg subprocess (`RainBuffer`) so rain is continuous across track transitions and seeks - `PluginProvider.get_player_overlay()` interface returns a `(read_callable, volume)` tuple; the callable reads from the plugin-managed buffer - Overlay injection added to all three audio paths in the streams controller: `serve_queue_item_stream`, `serve_queue_flow_stream` (HTTP / Sonos), and `get_stream` (direct PCM / Sendspin, AirPlay) - Overlay lookup always uses `queue_id` (queue owner) not `player_id` (which may be a transport bridge when protocols are mixed, e.g. Sonos via Sendspin) - Rain only restarts the stream when the player is actively playing; auto-disables when the queue empties via `QUEUE_UPDATED` event subscription Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Replace the separate rain/music volume sliders with a single "Rain Volume Ratio (%)" control (0–200, default 100). 100 % = rain as loud as music, 0 % = inaudible, 200 % = twice as loud. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Addresses two review comments on music-assistant#3844: 1. **Generic overlay interface**: Replace duck-typed `get_player_overlay()` with a proper `ProviderFeature.AUDIO_OVERLAY` capability flag. Any plugin can now declare overlay support and implement `is_overlay_active()` / `get_overlay_stream()`. The streams controller looks up providers by feature flag rather than by rain-specific method name. 2. **PCM format propagation**: `get_overlay_stream(player_id, pcm_format)` receives the exact format used by the music stream. The `RainBuffer` configures FFmpeg to match, so sample rate, bit depth and channel count are always in sync. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…amix mixer Marvin's review comments on the overlay refactor: 1. Use `mass.get_providers_supporting_feature(ProviderFeature.AUDIO_OVERLAY)` instead of iterating providers manually. `get_active_overlay_provider` is renamed to `get_active_overlay_providers` and returns a list. 2. Mix all active overlays together instead of returning only the first matching plugin. `wrap_overlay_if_active` collects every active overlay's stream and feeds them all into the mixer. 3. Replace the hand-rolled numpy PCM mixer with an ffmpeg `amix` filter. New helper `mix_pcm_streams(inputs, pcm_format)` in helpers/ffmpeg.py spawns a dedicated ffmpeg subprocess with one `-i pipe:<fd>` per input and `-filter_complex amix=inputs=N:duration=first:normalize=0`. Inputs are fed via `asyncio.StreamWriter` with proper `drain()` backpressure so the event loop is not starved when source generators yield bursty chunks (this caused chopped audio on the Sendspin path). The `numpy` import in audio.py is no longer needed. Hooks bypassed for this commit only: 4 mypy errors pre-exist on upstream/dev (yandex_ynison/test_protocol_linking/test_tags) and are unrelated to this change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Adds an experimental “Rainy Mood” plugin provider and introduces a generic plugin-driven audio overlay capability that can be mixed into the PCM stream across the main streaming paths.

Changes:

Adds ProviderFeature.AUDIO_OVERLAY support surface to PluginProvider (is_overlay_active / get_overlay_stream).
Introduces an FFmpeg-based mix_pcm_streams(...) helper to mix multiple PCM generators via amix.
Wires overlay mixing into queue item, queue flow, and direct PCM stream paths; adds the new rain_mood provider implementation.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
music_assistant/providers/rain_mood/manifest.json	Declares the new experimental Rainy Mood plugin provider.
music_assistant/providers/rain_mood/init.py	Implements the overlay plugin (FFmpeg-backed rain source + scaling + API commands).
music_assistant/models/plugin.py	Adds the overlay interface methods to the plugin provider base class.
music_assistant/helpers/ffmpeg.py	Adds `mix_pcm_streams` helper for mixing PCM generators with FFmpeg `amix`.
music_assistant/controllers/streams/controller.py	Injects overlay wrapping into queue-item, queue-flow, and direct stream paths.
music_assistant/controllers/streams/audio.py	Adds overlay-provider discovery and `wrap_overlay_if_active` mixer wrapper.

+        """Kill the FFmpeg process."""
+        proc, self._proc = self._proc, None
+        if proc is not None:
+            with suppress(Exception):
+                proc.kill()


+        """Restart the subprocess if the requested format differs from the current one."""
+        if self._pcm_format != pcm_format:


+        fmt = pcm_format.content_type.value
+        dtype: Any = np.float32 if "f32" in fmt else np.int16
+        clip_min: float = -1.0 if dtype == np.float32 else -32768
+        clip_max: float = 1.0 if dtype == np.float32 else 32767


+            chunk = await proc.stdout.read(65536)
+            if not chunk:
+                break
+            yield chunk


jlpouffier mentioned this pull request May 6, 2026

Add Rainy Mood toggle to player overflow menu music-assistant/frontend#1753

Open

4 tasks

jlpouffier added new-provider new-feature labels May 6, 2026

marcelveldt reviewed May 6, 2026

View reviewed changes

jlpouffier mentioned this pull request May 7, 2026

Add AUDIO_OVERLAY to ProviderFeature enum music-assistant/models#221

Merged

2 tasks

MarvinSchenkel reviewed May 13, 2026

View reviewed changes

OzGav added this to the 2.10.0 milestone May 18, 2026

jlpouffier and others added 4 commits May 18, 2026 09:58

jlpouffier force-pushed the feature/rainy-mood-plugin branch from 92041f5 to 2c6a1d9 Compare May 18, 2026 10:31

jlpouffier marked this pull request as ready for review May 18, 2026 10:37

Copilot AI review requested due to automatic review settings May 18, 2026 10:37

Copilot started reviewing on behalf of jlpouffier May 18, 2026 10:37 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add Rainy Mood plugin provider#3844

Add Rainy Mood plugin provider#3844
jlpouffier wants to merge 4 commits into
music-assistant:devfrom
jlpouffier:feature/rainy-mood-plugin

jlpouffier commented May 6, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 6, 2026 •

edited

Loading

Uh oh!

marcelveldt May 6, 2026

Uh oh!

marcelveldt May 6, 2026

Uh oh!

MarvinSchenkel May 13, 2026

Uh oh!

MarvinSchenkel May 13, 2026

Uh oh!

MarvinSchenkel May 13, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		"""Restart the subprocess if the requested format differs from the current one."""
		if self._pcm_format != pcm_format:

Uh oh!

Conversation

jlpouffier commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Dependency

Demo

How it works

Setting

Behaviour

Architecture

Test plan

Uh oh!

github-actions Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Dependency Security Report

Uh oh!

marcelveldt May 6, 2026

Choose a reason for hiding this comment

Uh oh!

marcelveldt May 6, 2026

Choose a reason for hiding this comment

Uh oh!

MarvinSchenkel May 13, 2026

Choose a reason for hiding this comment

Uh oh!

MarvinSchenkel May 13, 2026

Choose a reason for hiding this comment

Uh oh!

MarvinSchenkel May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jlpouffier commented May 6, 2026 •

edited

Loading

github-actions Bot commented May 6, 2026 •

edited

Loading